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-In-Service Training Programs Can Be Evaluated 


STANLEY J. HEYWOOD, Chairman, Teacher Education 
Coe College, Cedar Rapids, lowa 


Most school administrators today are 
encouraging and initiating in-service 
training programs for their teaching 
staffs. They and the teachers generally 
point to at least five reasons why in- 
service education is important: 


1. Teachers need to "keep up profes- 
sionally" in their work. New knowledge 
of educative processes is being devel- 
oped constantly; teachers must remain 
abreast of the developments in their 
field. 


2. Teaching is an increasingly complex 
task. Objectives and tasks change and 
increase as (a) society becomes more 
complex and (b) the major purposes of 
instruction are expanded to include not 
only the communication of facts but also 
the achievement of understanding and the 
development of skill in thoughtful 
analysis. 


3. Teachers today are faced with per- 
haps the heaviest demands ever made on 
them in inducting the young citizen into 
a constantly changing society which 
places maximum emphasis upon the worth 
and responsibility of the individual. 
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The translation of democratic principles 
into action is an imperative task of 
schools. 


4. Most teachers desire to develop 
more effective ways of dealing with in- 
structional problems. 


5. Many teachers in America today 
lack the professional qualifications for 
full certification by their respective 
states. In-service training is one means 
of gaining the qualifications needed. 


Types of In-Service Education 


A variety of means is employed in 
school systems to provide in-service 
training for teachers. They include con- 
ferences, institutes, workshops, faculty 
meetings, extension classes, correspond- 
ence courses, and study groups of vari- 
ous kinds. Almost any topic that can be 
said to relate to the improvement of 
education is deemed appropriate for con- 
sideration. Child study, curriculum 
revision, the educational philosophy of 
the school, school-community relations, 
and the preparation of resource units 
are but a few examples. 


How Effective Is It? 


Far more attention has been devoted to 
developing and conducting in-service 
training programs in school systems than 
has been given to measuring their effec- 
tiveness. In fact, appraisals have 
generally been confined to seeking the 
opinions of teachers and administrators 
as to how valuable they believed the 
programs to be. Admittedly, these 
opinions are useful; but they fail to 
provide objective evidence of profes- 
sional development in the teaching staff 
or of changes in the instructional pro- 
gram of a school as a result of in-ser- 
vice training. Without such evidence, 
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administrators and their staffs are 
never certain that the outcomes of such 
training justify the investment of time 
and money that is made. 


Identifying Objectives 

If the effectiveness of any activity 
is to be evaluated, its objectives must 
be known. In fact, the ?nitial planning 
of an in-service training program should 
be centered on determining the specific 
objectives which the school system hopes 
to achieve by the particular training 
program. Vague objectives to which lit- 
tle initial attention is given result 
usually in a vague program which is so 
difficult to evaluate that little ap- 
praisal is ever attempted. 


In a study completed recently by the 
writer as a staff member of the Midwest 
Administration Center, an effort was 
made to determine the specific objec- 
tives of the in-service training pro- 
grams in a limited number of schools and 
then to devise an experimental instru- 
ment which would measure progress toward 
the accomplishment of one of the objec- 
tives.! A team of three persons inter- 
viewed twelve administrators and thirty- 
five teachers in school systems located 
in the Greater Chicago area regarding 
the objectives of the in-service train- 
ing programs in their schools. Even be- 
fore all of them had been interviewed, a 
pattern of replies had been established 
which did not change as more persons 
were questioned. From the interviews, 
310 statements of objectives were ob- 
tained regarding ten different categor- 
ies such as teaching methods, prepara- 
tion of curriculum material, school-com- 
munity relations, ete. From the 310 
statements of objectives, twenty were 
selected which (1) aimed at a change in 
the behavior of the teachers involved 
and (2) were desirable and appropriate 
when examined in the light of current 
literature dealing with in-service 
training. 

The twenty objectives were placed in a 
questionnaire which was sent during 1953 
to all superintendents belonging to the 
Superintendent's Study Club, an organi- 
zation of administrators in the metro- 
politan area of Chicago. Each superin- 
tendent was asked to indicate which of 
the objectives, if any, were presently 
being used in a systematic way to guide 
the development of the in-service train- 
ing program in his school system. 


More than seventy administrators re- 
turned the questionnaire. Listed as an 
objective toward which thirty of the 
respondents were currently working was: 
"To develop in teachers a constructively 
critical attitude toward their own work! 
Because of the importance which the edu~ 
eational profession places on this ob- 
jective and in view of the fact that it 
is related directly to the improvement 
of instruction, it was selected as the 
one to be used in developing an instru- 
ment which could become one means of 
evaluating the effectiveness of in-ser- 
vice training programs pointed toward 
this objective. 


Developing the Instrument 


The first step in developing an in- 
strument to appraise growth in the 
direction of a broad objective is the 
development of a precise definition of 
the objective. Six superintendents who 
had indicated that the "development in 
teachers of a constructively critical 
attitude toward their work" was an ob- 
jective of their current in-service 
training programs were asked to work 
with the Midwest Administration Center 
staff in developing a detailed defini- 
tion of its meaning. The resulting 
statement indicated that a teacher who 
is constructively critical toward his 
own work (1) has clearly formulated pur- 
poses, (2) reviews his purposes system- 
atically and objectively, (3) evaluates 
objectively the activities in which he 
engages in terms of these purposes, and 
(4) revises or modifies activities in 
the light of the evaluation. 


The statement as agreed upon by the 
group indicated that the clear formula- 


tion of purposes is basic to the devel- 


opment of a constructively critical at- 
titude. Therefore, with the assistance 
of teachers, school administrators, 
advanced doctoral students, and a number 
of university professors, a question- 
naire entitled What Are Your Purposes? 
was developed. The questionnaire was 
intended to measure the extent to which 
teachers tend to accept, as their own, 
clear statements of purpose as opposed 
to vague statements. After testing in 
several schools and appropriate revi- 
sions, the questionnaire was adminis- 
tered by staff members of the Midwest 
Administration Center to 179 teachers in 
ten schools near Chicago in which ongo- 
ing in-service programs were attempting 
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to develop constructively critical atti- 
tudes on the part of teachers. The num- 
ber of teachers in these schools ranged 
from eight to forty. 


What Are Your Purposes? 


What Are Your Purposes? is a question- 
naire containing thirty-one questions.” 
Four possible answers are given for each 
question. The teacher may use one of 
these or write in his own. Typical of 
the questions and the answers are the 
following: 


When you use the discussion method of 
teaching your purpose is to: 


& use a method in which members of the 
class have a chance to discuss situa- 
tions growing out of the lesson. 


be use a method which will develop skill on 


the part of pupils in obtaining informa- 
tion. 


c use a method which will develop skill on 
the part of pupils in expressing their 
views and subjecting them to the ap- 
praisal of others. 


d. use a method which decreases the teach- 
er's responsibility for leading class 
activities. 


In this case the response that most 
closely corresponds to the definition of 
a clearly formulated purpose is response 
"ce". Response "a" is solely a restate- 
ment of what the discussion method actu- 
ally is. Response "b" is clearly not the 
objective of the discussion method. Re- 
sponse "d" would violate the philoso- 
phies of education of most teachers and 
administrators. The other thirty ques- 
tions cover a wide variety of classroom 
activities and have, following each of 
them, a statement of a clearly formu- 
lated purpose, a simple restatement, a 
statement which is not an objective or 
purpose, and a statement which is con- 
trary to modern conceptions of educa- 
tion. A key consisting of the preferred 
response to each of the thirty-one items 
was developed with the assistance of a 
panel of competent judges. 

The questionnaire was found to have a 
sufficiently high degree of reliability 
to justify its use as a measure of group 
achievement. Its "face validity" was 
established by the procedure through 
which the instrument was developed. More 
important is the fact that a test in one 
school showed that there was a reason- 
able agreement between the predictions 
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of trained observers as to how teachers 
would respond to the questions and the 
actual responses made by the teachers. 
(The predictions were based on observa- 
tions made of the teachers in their 
classroom work.) 


Purposes, Clear and Vague 


Analyses of the responses made by the 
179 teachers in the ten schools showed 
that two-thirds or more had clearly 
formulated purposes in regard to: 
(1) taking pupils on field trips, 
(2) the use of teacher-made tests, 
(3) achieving reasonable order and 
discipline, (4) group project work, 
(5) grouping pupils, (6) the discussion 
method of teaching, (7) guidance on per- 
sonal problems, and (8) practice methods. 
On the other hand, 50 per cent or less 
had clearly formulated purposes in 
(1) the appraisal of progress of indi- 
vidual pupils, (2) the use of pictorial 
material, (3) the use of films, (4) 
choosing learning experiences for pupils, 
(5) making decisions as to teaching 
methods to be used, (6) making studies 
of pupils as a basis for curriculum 
planning, and (7) making a class in- 
creasingly responsible for planning and 
carrying out its own activities. 


Teachers varied greatly in the number 
of their responses which were in accord 
with those considered best by the panel 
of judges. Percentage scores of indi- 
vidual teachers, obtained by dividing 
the number of correct answers by the 
number of questions attempted, ranged 
from 12.5 per cent to 92.8 per cent. 
Wide variations were found also in the 
responses when the various schools par- 
ticipating in the study were compared. 
Supplementary information obtained from 
the schools tended to show, as might be 
expected, that the more specifically 
schools were focusing on problems re- 
lated to the determination of instruc- 
tional objectives the more clarity there 
was in the purposes of the teachers. 
This generalization is being tested by 
analyses of additional data collected 
during the past school year. 


It Can Be Done in Other Schools 


The study attempted to apply to the 
problem of evaluating in-service train- 
ing activities the same general proce- 
dures commonly used by teachers in ap- 
praising pupil achievement.* It did not 
presume to evaluate all aspects of even 


one program, however; the objective was 
to demonstrate a procedure in planning 
for evaluation and to develop an experi- 
mental instrument such as any school 
system might devise to measure the ac- 
complishments of its in-service training 
programs. In the study, of course, in- 
structional activities were located in 
which teachers appeared to have clearly 
formulated purposes. Discovered also 
were activities in which their purposes 
appeared quite vague despite the fact 
that the in-service training programs 
had supposedly given attention to them. 
Administrators and teachers everywhere 
might well examine their own activities 
in those areas where vague purposes were 
discovered. Observations made in school 
systems other than those included in 
this study appear to indicate similar 
vagueness of purpose in some of the same 
areas. 

Regardless of the approach or the 
instruments used, sufficient evidence is 
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now at hand to demonstrate the impor- 
tance of defining specific objectives 
for in-service training programs. The 
determination of these objectives by the 
school staff during the initial planning 
of in-service programs will do much to 
assure the success of the work that is 
undertaken. Furthermore, the procedure 
is essential if subsequent appraisal is 
to be made of the progress, if any, 
which results from the program. 


Istanley J» Heywood, "Determining Clarity of 
Teachers’ Purposes." Unpublished Ph.D. disser- 
tation, Department of Education, University of 
Chicago, 1954. 


“Readers may obtain sample copies of the 
questionnaire, while the supply lasts, by 
writing to the Midwest Administration Center. 

3phis method is summarized concisely in: 
Ralph W. Tyler, Basic Principles of Curricu- 
lum_and Instruction. Chicago: University of 
Chicago Press, 1950. 


1. How should you and your staff decide on really specific objectives for 
your in-service training programs? 


2. What specific steps can you and your staff take at this time to deter- 
mine the value of your current in-service training programs? 


3. What assistance, if any, would your teachers derive from examining criti- 
cally the purposes behind their present classroom activities? 


Unjp Ss 


ve, 


3 ‘ \| @ 

£ 

JAN § 
Library 


a 
¢ 
§ 
/ 


; 


